CONCENTRATION OF MEASURE ON PRODUCT SPACES WITH 
APPLICATIONS TO MARKOV PROCESSES 



GORDON BLOWER AND FRANCOIS BOLLEY 

Abstract. For a stochastic process with state space some Polish space, this paper gives 
sufficient conditions on the initial and conditional distributions for the joint law to satisfy 
Gaussian concentration inequalities and transportation inequalities. In the case of the 
Euclidean space R m , there are sufficient conditions for the joint law to satisfy a logarithmic 
Sobolev inequality. In several cases, the obtained constants are of optimal order of growth 
with respect to the number of random variables, or are independent of this number. 
These results extend results known for mutually independent random variables to weakly 
dependent random variables under Dobrushin-Shlosman type conditions. The paper also 
contains applications to Markov processes including the ARMA process. 



I. Introduction 

Given a complete and separable metric space (X,d), Prob(X) denotes the space of 
Radon probability measures on X, equipped with the (narrow) weak topology. We say 
that \x G Prob (X) satisfies a Gaussian concentration inequality GC(k) with constant k on 
(X, d) if 

/ exp(tF(x))u.(dx) < expft / F(x) n{dx) + /tt 2 / 2 ) (teR) 

holds for all 1-Lipschitz functions F : (X, d) — > IR (see Recall that a function g : 

(tti, d\) — > (f2 2 , d 2 ) between metric spaces is L-Lipschitz if d 2 (g(:r), g{y)) < L di(x, y) holds 
for all x, y G fii, and we call the innmum of such L the Lipschitz seminorm of g. 

For k > 1 and X\,...,Xk in X, we let = {x\,...,Xk) G X k and, given 1 < 
s < oo, we equip the product space X k with the metric d^ defined by d^ s \x^ k \y^) = 
{Y^j=id{xjiyj) s ) l/s for and y (fc) in X k . 

Now let (£j)"=i be a stochastic process with state space X. The first aim of this paper 
is to obtain concentration inequalities for the joint distribution P' n ' of = (£i, . . . 
under hypotheses on the initial distribution of £i and the conditional distributions 
Pk(. | x^ 1 )) of £fc given we recall that is given by 

P( n \dx {n) ) = Pn {dx n | X (n " 1} ) . . . P2 (dx 2 | X X ) P (1) (rfXi). 
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If the (£j)j=i are mutually independent, and the distribution of each ^- satisfies GC(k), 
then P^ on (J n ,ciW) is the product of the marginal distributions, and inherits GC(nn) 
from its marginal distributions by a simple 'tensorization' argument. A similar result also 
applies to product measures for the transportation and logarithmic Sobolev inequalities 
which we consider later; see [T21 I23| . To obtain concentration inequalities for P^ when 
are weakly dependent, we impose additional restrictions on the coupling between the 
variables, expressed in terms of Wasserstein distances which are defined as follows. 

Given 1 < s < oo, Prob s (X) denotes the subspace of Prob(X) consisting of v such 
that j x d(xQ,y) s u(dy) is finite for some or equivalently all Xq G X. Then we define the 
Wasserstein distance of order s between \i and v in Prob.,(X) by 

W.(ji,v) = M(ff d(x,y) s Tr(dxdy)y /S (1.1) 

where 7r e Prob s (X x X) has marginals tti = fi and tt 2 = v. Then W s defines a metric on 
Prob s (X), which in turn becomes a complete and separable metric space (see pUl 

In section 3 we obtain the following result for time-homogeneous Markov chains. 

Theorem 1.1. Let (^)™ =1 be an homogeneous Markov process with state space X, initial 
distribution P^ and transition measure p(. \ x). Suppose that there exist constants K\ and 
L such that: 

(i) and p(. \ x) (x E X) satisfy GC(k{) on (X,d); 

(ii) x h- > p(. | x) is L-Lipschitz (X, d) —* (Probi (X), Wi). 

Then the joint law P( n > of . . . , ^ n ) satisfies GC(K n ) on (X n , S 1 '), where 

n m—1 2 
m=l k=0 

In Example 16 .31 we demonstrate sharpness of these constants by providing for each value 
of L a process such that K n has optimal growth in n. 

Concentration inequalities are an instance of the wider class of transportation inequali- 
ties, which bound the transportation cost by the relative entropy. We recall the definitions. 

Let v and /i be in Prob(X), where v is absolutely continuous with respect to //, and let 
du/du. be the Radon-Nikodym derivative. Then we define the relative entropy of v with 
respect to \i by 

dv 



Ent(i/ I u) — \ log — dv\ 
Jx do. 

note that < Ent(z/ | ^i) < oo by Jensen's inequality. By convention we let Ent(^ | //) = oo 
if v is not absolutely continuous with respect to ji. 

Given 1 < s < oo, we say that /i G Prob s (X) satisfies a transportation inequality T s (a) 
for cost function d(x,y) s , with constant a, if 

W,(v,n) < (^Ent(z/ | /z)^ 1 '" 
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for all v G Prob s (X). 

Marton ^3] introduced T 2 as 'distance-divergence' inequalities in the context of informa- 
tion theory; subsequently Talagrand [2B! showed that the standard Gaussian distribution 
on R m satisfies T^l). Bobkov and Gotze showed in |2j that GC(k) is equivalent to Ti(l//c); 
their proof used the Kantorovich-Rubinstein duality result, that 



where //, v G Probi(X) and / runs over the set of 1-Lipschitz functions / : X — > R. A 
v G Prob(X) satisfies a T\ inequality if and only if v admits a square-exponential moment; 
that is, f x exp(f3d(x, y) 2 ) u(dx) is finite for some f3 > and some, and thus all, y G X; 
see E| for detailed statements. Moreover, since T s (a) implies T r (a) for 1 < r < s by 
Holder's inequality, transportation inequalities are a tool for proving and strengthening 
concentration inequalities; they are also related to the Gaussian isoperimetric inequality 
as in [2]. For applications to empirical distributions in statistics, see [H8J . 

Returning to weakly dependent (£j)™ =1 with state space X, we obtain transportation 
inequalities for the joint distribution P^ n \ under hypotheses on and the conditional 
distributions. Djellout, Guillin and Wu developed Marton's coupling method O EH] 
to prove T 3 (a) for under various mixing or contractivity conditions; see also (221, or 
[H] where the conditions are expressed solely in terms of exponential moments. We extend 
these results in sections |5] and Elbelow, thus obtaining a strengthened dual form of Theorem 



Theorem 1.2. Let (^)™ =1 be an homogeneous Markov process with state space X , initial 
distribution P^ and transition measure p(. \ x). Suppose that there exist constants 1 < 
s < 2, a > and L > such that: 

(i) P^ and p(. \ x) (x G X) satisfy T s (a); 

(ii) x h- ► p(. | x) is L-Lipschitz (X, d) — > (Prob s (X), W s ). 

Then the joint distribution P^ of (^i, . . . , £ n ) satisfies T s (a n ), where 



in particular independent of n for s = 2 when L < 1 . 

Our general transportation Theorem 12.11 will involve processes that are not necessarily 
Markovian, but satisfy some a hypothesis related to Dobrushin-Shlosman's mixing con- 
dition [8, p. 352; 15, Definition 2}. When X = IR m , we shall also present some more 
computable version of hypothesis (ii) in Proposition I2.2[ and later consider a stronger 
functional inequality. 





if L < 1, 
if L = l, 

if L>1; 
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A probability measure fi on 
constant a > if 



satisfies the logarithmic Sobolev inequality LSI (a) with 



/ 2 log(/ 2 / / / 2 d/iW<- / 



\\vf\\%du. 



holds for all / G L 2 (dfi) that have distributional gradient V/ G L 2 (dfi;. 



m , let ||(a fc )||/« = (5Xj l a fcH 1/s for 1 < s < cx), and 
The connection between the various inequalities is summarized by 

LSI(a) T 2 (a) => T x {a) GC(l/a); 



sup 1<fc<r 



Given (a*.) G 
(1.2) 



see [3; 18; 24, p. 293]. Conversely, Otto and Villani showed that if fi(dx) 
satisfies T 2 (a) where V : M m — > M is convex, then \x also satisfies LSI (a /4) (see [4; 18; 24, 
p. 298]); but this converse is not generally true, as a counter-example in [U] shows. 

Gross ^T] proved that the standard Gaussian probability measure on R m satisfies LSI(1). 
More generally, Bakry and Emery PQ showed that if V is twice continuously differentiable, 
with Hess V > al m on IR m for some a > 0, then fx(dx) = e~ v ^ dx satisfies LSI (a); see 
for instance [221 for extensions to this result. Whereas Bobkov and Gotze characterized 
in terms of their cumulative distribution functions those \x G Prob(R) that satisfy LSI (a) 
for some a, there is no known geometrical characterization of such probability measures 



on 



when m > 1. 



Our main Theorem 15 .11 gives a sufficient condition for the joint law of a weakly dependent 
process with state space M. m to satisfy LSI. In section |B] we deduce the following for 
distributions of time-homogeneous Markov processes. Let d/dx denote the gradient with 
respect to x G M m . 

Theorem 1.3. Let (£j)T=i be an homogeneous Markov process with state space W 71 , initial 
distribution and transition measure p(dy \ x) = e~ u ^ x ' y ^dy. Suppose that there exist 
constants a > and L > such that: 

(i) andp(. | x) (x G M m ) satisfy LSI (a); 

(ii) u is twice continuously differentiable and the off-diagonal blocks of its Hessian matrix 
satisfy 

d 2 u 

< L 



dxdy 

as operators (R m ,£ 2 ) -> (M m ,£ 2 ). 

Then the joint law of the first n variables (£i, 

(a - L) 2 



,£ n ) satisfies LSI (a n ), where 



a, 



a 



a 



n(n + l)(e- 1) 
a\ 2n L 2 -a 2 

L 



if 
if 



L < a, 
L = a, 



ae(n + 1) 

in particular a n is independent of n when L < a. 



if L > a; 
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The plan of the paper is as follows. In section El we state and prove our results on 
transportation inequalities, which imply Theorem II .21 and in section|3]we deduce Theorem 
11.11 In section 0] we prove LSI(a) for the joint distribution of ARMA processes, with a 
independent of the size of the sample. In sectionElwe obtain a more general LSI, which we 
express in a simplified form for Markov processes in section |3 Explicit examples in section 
H3 show that several of our results have optimal growth of the constants with respect to n 
as n — > oo, and that the hypotheses are computable and realistic. 



2. Transportation inequalities 

Let (Ck)k=i ^ e a stochastic process with state space X, let p k (- \ x^ h ~^) denote the 
transition measure between the states at times k — 1 and k, and let be the joint 
distribution of Our main result of this section is a transportation inequality. 

Theorem 2.1. Let 1 < s < 2, and suppose that there exist a\ > and M > > 
{I — 1, . . . , n) such that: 

(i) PW andp k (. | xC*" 1 )) (jfc = 2, . . . ,n; x^ 1 ^ E X^ 1 ) satisfy T s {a) on {X,d); 

(ii) x^ 1 ^ I— > pk(- | x^ 1 ^) is Lipschitz as a map (X fc_1 ,d ( - S ^) — > (Prob s (X), W s ) for 
k = 2, . . . , n, in the sense that 

fe-i 

W s (p k (. | x^),p k (. | y^)) s < J2Pk-jd(xj,yiY (x (fc - 1) ,y (fc " 1) e X k ~ 1 ). 

3=1 

Then satisfies the transportation inequality T s (a n ) where 



a„ = a 



[1 + M)' 



a, 



if R>1. 



Suppose further that 
(m) EJ=iPj < R - 
Then the joint distribution p( n > satisfies T s (a n ) where 

n 1- ( 2 / s )(l — B}/ S ) 2 a if R< 1 

e (2/.)-2( n+1 )-(2/.)-l if R=l 

R-iy/' a 
.e^HV n + 1 

In hypothesis (iii), the sequence (pk)kZi measures the extent to which the distribution 
of £ n depends upon the previous £„_i, ^ n -2, . . . ; so in most examples (pk) k Zi is decreasing. 

A version of Theorem 12.11 was obtained by Djellout, Guillin and Wu, but with an explicit 
constant only when R < 1; see Theorem 2.5 and Remark 2.9]. Theorem 12.11 also 
improves upon section 4 of 0, where the assumptions were written in terms of moments 
of the considered measures. 

The Monge-Kantorovich transportation problem involves finding, for given //, v 6 Prob(X), 
an optimal transportation strategy in (jl.lj) . namely a 7r that minimises the transportation 
cost; a compactness and semi-continuity argument ensures that, for suitable cost functions, 
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there always exists such a 7T. We recall that, given \i 6 Prob(X), another Polish space Y 
and a continuous function ip : X —>■ Y, the measure induced from /z by <£> is the unique 
G Prob(F) such that 

/ f(y)u(dy) = [ f(<p(x))ii(x) 

for all bounded and continuous / : X — > R. Brenier and McCann showed that if fi and v 
belong to Prob2(M m ), and if moreover [i is absolutely continuous with respect to Lebesgue 
measure, then there exists a convex function $ : K' m — > M. such that the gradient <p = V$ 
induces \i from v and gives the unique solution to the Monge-Kantorovich transportation 
problem for s = 2, in the sense that 

/ ||V$(x) - jz(dz) = W 2 {^ ) v) 2 . 

Further extensions of this result were obtained by Gangbo and McCann for 1 < s < 2, 
by Ambrosio and Pratelli for s = 1, and by McCann in the context of compact and 
connected C 3 -smooth Riemannian manifolds that are without boundary (see also [7[ I24j ) . 

Proof of Theorem \2.1\ In order to give an explicit solution in a case of importance, we 
first suppose that X = IR m and that P^ and Pj(dxj \ x^^ 1 ') (j = 2,...,n) are all 
absolutely continuous with respect to Lebesgue measure. Then let Q< n ) e Prob s (M" m ) be 
of finite relative entropy with respect to p( n \ Let Q^\dx^) be the marginal distribution 
of x^ E W m with respect to Q^(dx^), and disintegrate in terms of conditional 
probabilities, according to 

Q(J\dx^) =q j (dx j | x^)Q^- x \dx^). 

In particular qj(. \ x^~^) is absolutely continuous with respect to pj(. \ x^~ x ') and hence 
with respect to Lebesgue measure, for Q^' 1 ' almost every x^~ l > . A standard computation 
ensures that 



Ent (Q {n) | P (ri) ) = Ent | P (1) ) (2.1) 

n „ 

+ J2 / Ent te( • I xij ~ 1] ) I pA • I xU ~ 1} )) Q ij ~ l \dx {j - i) ). 

• =2 JK.^- 1 )™- 

When the hypothesis (i) of Theorem 2.1 holds for some 1 < s < 2, it also holds for s — 1. 
Consequently, by the Bobkov-Gotze theorem, P^ and Pj(dxj \ x^~^) satisfy GC(k) for 
K = 1/ct, and then one can check that there exists e > such that 

exp( £ ||x (1) ||, 2 2 )P (1) (da; (1) ) < oo 

and likewise for pf compare with Herbst's theorem [2U p. 280], and [cOIH]- Hence and 
qj(dxj | x^" -1 )) for Q^" 1 ' almost every x^~^ have finite second moments, since by Young's 
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inequality 

e\\x {n) \\lQ {1 \dx^) < Ent(Q« | P (1) ) + log ! exp(e\\x^\\l)P^(dx^) < oo 

and likewise with qj and in place of and P^ respectively. 

Let Q\ : IR m — > M m be an optimal transportation map that induces P^-^dxi) from 
Q^\dxi); then for every each xi, let x 2 i— > #2(^1, £2) induce p 2 (dx 2 | ^(a^)) from 
q 2 (dx 2 I 37) optimally; hence 0^ : M 2m — > M 2m , defined by 

0^ 2 ^(xi,X2) = (6'i(xi), #2(^1, X2)) on a certain set of full measure, induces P^ from 
Q( 2 \ Generally, having constructed 0^) : W m — > W m , we let Xj + i 1— > 6 , J+1 (a;^\ be an 
optimal transportation map that induces pj + ±(dxj + i | Q^'(x^')) from qj + i(dxj + i \ x^), 
for all in a certain set of full measure; then we let 0( J ' +1 ) : R(f+i)"» Ri m x E m 
be the map defined by 

0Cj+i)(a;(?+ 1 )) = (e^^W),^!^^ 1 ')) 

on a set of full measure. In particular 0(-? +1 ) induces p( J+1 ) from Q^ +1 >, in the style 

of Kneser. 

This transportation strategy may not be optimal, nevertheless it gives the bound 

„ n 

W s (Q {n \ P (n) ) s < / ||0 (n) (a; (n) ) -x {n) \\lsQ (n \dx {n) ) = ^d k (2.2) 
^ Rnm k=i 

by the recursive definition of @( n \ where we have let 

d k = [ \\6 k (xW)-x k \\ s is QW(dxW) (k=l,...,n). 

JR km 

However, the transportation at step k is optimal by construction, so 

d k = f W s {p k (. I e^\x^)),q k (. I x^)) s Q^\dx^). (2.3) 



[k-l)m 



Given a, b > 0, 1 < s < 2 and 7 > 1, we have (a + b) s < (7/(7 — l)) s 1 a s + j s 1 6 s . Hence 
by the triangle inequality, the expression ()2.3|) is bounded by 



7 \ 8—1 



7-1 



m(fe-l) 



+ 7 S_1 / W,(P*(. I (fc - 1) (x (fc - 1) )),p fe (. I x (fc - 1) )) i Q (fc - 1) (dx (fc - 1) ). (2.4) 



By hypothesis (i) and then Holder's inequality, we bound the first integral in ()2.4j) by 

/ ry \s-i/2\s/ 2 / /" \ s / 2 

fc* = (— L -r) - / Entfe | Pfc )rfg( fc - 1)N 

V7-I/ \aJ \J R (k-l)m 
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Meanwhile, on account of hypothesis (ii) the second integral in (|2.4|) is bounded by 

. fc-i fc-i 
7 s " 1 / TP^W 9 ^) - x i\\" Q (k ' l \dx^) =7'- 1 52pk- j d j , 

JlHm(k-l) ^ ^ 

and when we combine these contributions to (|2.4jl we have 

fc-i 

dk < h k + Y' 1 ^2pk- j d j . (2.5) 
j=i 

In the case when the pe are merely bounded by M, one can prove by induction that 

fc-i 

d k <h k + 7 S ~ 1 M ^2hj{l + Y^M)"- 1 - 3 , 
i=i 

so that 

J> < J^h^l + j^M)^ < (jThf s ) S/2 (^(l + 7 s - 1 M) 2 ^)/( 2 - s )) (2 " S)/2 
fc=i j=i j=i j=i 

by Holder's inequality. The first sum on the right-hand side is 

(i>f )* /2 = (^i)"'(l)* /2Ent w w i p( " ) )* /2 

by (|2.1|) . Finally, setting 7 = 1 + 1/n, we obtain by (|2.2|) the stated result 
W S (Q^,P^) S < ^y/ 2 (j_+MT ( ne )^-i En t(Q^ I P("») s / 2 . 

(iii) Invoking the further hypothesis (iii), we see that T m = X^jli dj satisfies on account 
of ()2.5|) the recurrence relation 

m+1 

T m+1 <J2 h 3+Y~ lRT m, 

3=1 

which enables us to use Holder's inequality again and bound T n by 

n k n n—j 

k=l j=l j=l 1=0 



j=i i=i £=0 



for 1 < s < 2. By ()2.2|) and the definition of T n this leads to 



n m—1 



m=l e=o 

n-1 



< (^_j S ^i-/ 2 ^^-^)^ (-Ent(Q< n > | P (n) )) S/2 ; (2.7) 

this also holds for s = 2. Finally we select 7 according to the value of R to make the bound 
(12. 7j) precise. When P < 1, we let 7 = R~ 1/s > 1, so that 7 s " 1 /? = R 1/s < 1, and we 
deduce the transportation inequality 

w s (q^,p^y < (l) s/ \ 1 n '^l y E ^(Q {n) 1 p(n) r /2 - 

When R > 1, we let 7 = 1 + 1/n to obtain the transportation inequality 

w.(Q<»\ p<»>)' < ^y /2 (n + 1)s -i n w 2 ( ^^^P^ 1 ) Ent ^ (n) I pin) Y /2 > 

which leads to the stated result by simple analysis, and completes the proof when X = M m . 

For typical Polish spaces (X,d), we cannot rely on the existence of optimal maps, but 
we can use a less explicit inductive approach to construct the transportation strategy, as 
in Given j = 1, . . . , n — 1, assume that 71"^ E Prob(X 2:; ) has marg inals QW(dxW) and 
P^(dy^) and satisfies 



W S (Q^,P^Y< [ y2d(x k ,y k ) s ^\dx^dy^ 



..,.„...,„ ^ ^ 

k=l 

Then, for each [x^\y^') E X 2j , let 0"j+i(. | x^\y^) E Prob(X 2 ) be an optimal trans- 
portation strategy that has marginals qj + i(dxj + i \ x^>) and Pj+i(dyj + i \ yV>) and that 
satisfies 

U)\ n . J I „UhY = / rl( , (An-. . , rim. . , I rJS) , ( 0')^ 



I x [3) ),Pj+i( ■ \ y U) )) s = / d(ar i+1 ,2/ i+ i)*cr i+ i(da; i+ idj/ i+1 | x^^y^'). 

Jx 2 

Now we let 

n {j+1 \dx u+1) dy {j+1) ) =a J+1 (dx j+1 dy j+1 \ x {j \y {j) ) 7i {j \dx U) dy (l) ), 

which defines a probability on X 2 ^ +1 ' with marginals Q^ +1 '(dx^ +1 ') and P^ +1 '(dy^ +1 '). 
This may not give an optimal transportation strategy; nevertheless, the recursive definition 
shows that 

n „ 

and one can follow the preceding proof from (|2.2j) onwards. □ 
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Proof of Theorem 1.2. Under the hypotheses of Theorem 11.21 we can take p\ — L and 
Pj = for j = 2, . . . , n, which satisfy Theorem 12. II with R = L in assumption (iii). □ 

The definition of W s not being well suited to direct calculation, we now give a com- 
putable sufficient condition for hypothesis (ii) of Theorem 12.11 to hold with some constant 
coefficients p e when (X,d) = (R m ,£ s ). 

Proposition 2.2. Let Uj : W m —>■ R be a twice continuously differentiable function that 
has bounded second-order partial derivatives. Let 1 < s < 2 and suppose further that: 

(i) Pj(dxj | x^~ 1 ') = exp(—Uj(x^)) dxj satisfies T 3 {a) for some a > and all x^~ x > E 
r«*Cj'-i) ; 

(ii) there exists some real number M s such that 

'du.i\j- 1 



sup 

xO- 1 ) 



J k=l I s ' 



dXk' k=l 



where l/V + l/s = l and d/dxk denotes the gradient with respect to Xk- 
Thenx( j -V i-> Pj {. | x^) is (M s / a) -Lipschitz (I m ' J '" 1, ,f) -> (Prob s (R m ), W s ). 

Proof. Given x {j ~ 1] E f"^ 1 ', we let x^(t) = (1 - t)x^ + tx^- 1] (0 < t < 1) 

be the straight-line segment that joins them, and we consider 

f(t)=W s ( Pj (.\x^(t)), Pj (.\x^)); 

then it suffices to show that / : [0, 1] — ► R is Lipschitz and to bound its Lipschitz seminorm. 
By the triangle inequality and (i), we have 

f(t + 6)-f(t) y 

5 ) 

< ^W s ( Pj (. | x^(t + 8)), Pj (. | x^(t))) 2 



< 

a5 2 



)-\Ent( Pj (. | xV-Vfr+S)) | p 3 (. | x^- 1 \t)))+Ent(p j (. \ x^(t)) | Pj {. \ x^(t+S))) 



a5 2 



{u^x^it + S),xj) - UjixV-V^^j)) 

{exp(-M i (x (j - 1) (t),x j )) - exp(-M i (x (i - 1) (t + 5),^))} dxj. (2.8) 
However, by the assumptions on Uj and the mean-value theorem, we have 

Uj (X {j - l) (t + 6), Xj ) - Uj (X ij - 1} (t) , Xj) 

= 6^2(j^-(x^- 1 \t),x j ),x k - x k ) + -(Ressuj (a^-D - x^~\ (x^ - x<* -*>)), 
fc=i k 
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where HessWj is computed at some point between (x^ and (x^ and is uni- 

formly bounded. Proceeding in the same way for the other term ([2.8)1 . we obtain 



lim sup 

5^0+ 



f{t + 6)-f(t) \' 
5 J 



< 



" k=1 OX k ' 



Hence by Holder's inequality we have 
\f(t + 8)-f(t)\ 



lim sup 



5 



< 



(t),Xj) 



s'\ 2/s' 



1/2 



for 1 < s < 2, and likewise with obvious changes for s — 1. By assumption (ii) and Vitali's 
theorem, / is Lipschitz with constant \J (M s / otjWx^^ — as required. □ 



3. Concentration inequalities for weakly dependent sequences 
In terms of concentration inequalities, the dual version of Theorem 12 . II reads as follows. 

Theorem 3.1. Suppose that there exist K\ > and M > pj > (j = 1, . . . ,n) such that: 
(i) P« and p k (. \ x^~^) (k = 2, . . . , n; x^' 1 ^ e X^ 1 ) satisfy GC7(« X ) on (X, d); 
(wj x^ 1 ) i — > p k (. | x^ -1 )) is Lipschitz as a map d^) — > (Prob\ (X), W\) for 

k — 2, . . . , n, in the sense that 

k-l 

Wx(p k (. | x^), Pk (. | y^)) < Y,Pk-jd(x„y,) (x^, y^ G X k ~ l ). 

3=1 

Then the joint law P^ satisfies GC(n n ) on (X n ,d^ 1 ' > ), where 

(1 + M) 2n 

K n ~ «1 Jj2 • 

Suppose moreover that 
(in) YTj=iPj < R - 
Then P (n) satisfies GC{k u (R)) on (X n ,d^), where 

yi vn — 1 

K n (R) = K 1 j2(J2 Rk ) 2 - 

m=l k=0 

Proof of Theorem 3.1. This follows from the Bobkov-Gotze theorem [Hj and the bound 
()2.6|) with s = 1 in the proof of Theorem 12.11 □ 



Alternatively, one can prove Theorem 13.11 directly by induction on the dimension, using 
the definition of GC. 
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Proof of Theorem 1.1. Under the hypotheses of Theorem 11.11 we can apply Theorem 13.11 
with p\ — L and pj = for j — 2, . . . , n, which satisfy (iii) □ 



4. Logarithmic Sobolev inequalities for ARMA models 

In this section we give logarithmic Sobolev inequalities for the joint law of the first n 
variables from two auto-regressive moving average processes. In both results we obtain 
constants that are independent of n, though the variables are not mutually independent, 
and we rely on the following general result which induces logarithmic Sobolev inequalities 
from one probability measure to another. For m > 1, let u G Prob(M m ) satisfy LSI(a), 
and let tp be a L-Lipschitz map from (M m , £ 2 ) into itself; then, by the chain rule, the 
probability measure that is induced from v by <p satisfies LSI(a/L 2 ). Our first application 
is the following. 

Proposition 4.1. Let Zq and Yj (j = 1, 2, . . . ) be mutually independent random variables 
in W™, and let a > be a constant such that the distribution of Zq and the distribution 
of Yj (j = 1,2,...; satisfy LSI (a) . 

Then for any L-Lipschitz map B : M. m — > MJ 71 , the relation 

Z j+1 = e(Z j )+Y j+1 (j = 0,l,...) (4.1) 

determines a stochastic process such that, for any n > 1, the joint distribution p( n_1 ) of 
(Zj)™~q satisfies LSI(a n ) where 

( (1 _ T\2, 



a,. 



[l-Lya if 0<L<1, 

if L = l, 

if L>1. 



n(n + l)(e - 1) 
L — l a 

L n e(n + 1) 



Proof. For (z , j/i, . . . , y n -i) e M nm , let <p n (z , j/i, . . . , y n -i) be the vector (z , . . . , 2„-i), 
defined by the recurrence relation 

z k +i = B(z k ) + y k+1 (k = 0,...,n-2). (4.2) 

Using primes to indicate another solution of (J4.2j) . we deduce the following inequality from 
the Lipschitz condition on G: 

||z fc+1 - 4+if < (1 + s)L 2 \\z k - 4|| 2 + (1 + e-^h^ - y' k+1 \\ 2 (4.3) 

for all e > 0. In particular ()4.3|) implies the bound 

k 

\\*k ~ 4f < ((1 + e)L 2 ) k \\z - z'J 2 + (1 + s- 1 ) J> + eW^WVi ~ v'^- 

3=1 
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By summing over k, one notes that (p n defines a Lipschitz function from (M. nm ,£ 2 ) into 
itself, with Lipschitz seminorm 



"-' 1/2 



^<((i+0£((i+^ 2 ) fc ) 



k=0 

We now select e > according to the value of L: when L < 1, we let e = L^ 1 — 1 > 0, so that 
< (1— whereas when L > 1, we let e = n" 1 , and obtain < [n(n+l)(e— l)]^ 1 / 2 ) 
for L = 1, and L Vn < [e(n + l)L n (L - I)- 1 } 1 ' 2 for L > 1. 

Moreover, y? n induces the joint distribution of (Zj)™~l from the joint distribution of 
(Z , Yi, . . . , Yn-i)- By independence, the joint distribution of (Z , Yi, . . . , Y n _i) is a product 
measure on (R nm , £ 2 ) that satisfies LSI (a). Hence the joint distribution of (Zj)™~q satisfies 
LSI (a), where a = L~ 2 a. □ 

The linear case gives the following result for ARMA processes. 

Proposition 4.2. Let A and B be m x m matrices such that the spectral radius p of 
A satisfies p < 1. Let also Z and Yj (j = 1,2,...) be mutually independent standard 
Gaussian N(0, I m ) random variables in W m . Then, for any n > 1, the joint distribution of 
the ARMA process (Zj)™I , defined by the recurrence relation 

Z J+l = AZ J+ BY J+l (j = 0,l,...), 

satisfies LSI {a) where 

y/P) 



a 



( V/Vn-.*|M.'||2 



-2 



Proof. By Rota's Theorem ^H], A is similar to a strict contraction on (M m , £ 2 ); that is, there 
exists an invertible mxm matrix S and a matrix C such that ||C|| < 1 and A = ^TpS^C S; 
one can choose the similarity so that the operator norms satisfy 

oo 

< ^p^'ll^'n 2 < oo. 

3=0 

Hence the ARMA process reduces to the solution of the recurrence relation 

SZ J+1 = ^TpCSZ, + SBY 3+1 (j = 0,1,...) (4.4) 

which involves the y/p-Lipschitz linear map G : ~R m —>■ W 71 : Q(w) = ^fpC w. Given n > 1, 
the linear map $ n : R nm — > R nm , defined to solve ()4.4j) by 

(z , ?/i, ... , y n ) ^ (Sz , SByx, SByn^x) h-> (Sz , Sz h Sz n -i) h-> (z , zi, • • • , Zn-i), 
has operator norm 

||*»|| < I^IHI^IKl-^^maxllJIfill}; 
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moreover, $ n induces the joint distribution of (Z , . . . , Z n _i) from the joint distribution of 
(Z ,Yi, . . . , F n _i). By Gross's Theorem (see [H]), the latter distribution satisfies LSI(1), 
and hence the induced distribution satisfies LSI(a), with a = ||$ n || -2 . □ 

Remarks 4.3. (i) As compared to Proposition 14. 1[ the condition imposed in Proposition 
14.21 involves the spectral radius of the matrix A and not its operator norm. In particular, 
for matrices with norm 1, Proposition 14. II only leads to LSI with constant of order n~ 2 \ 
whereas Proposition 14.21 ensures LSI with constant independent of n under the spectral 
radius assumption p < 1. 

(ii) The joint distribution of the ARMA process is discussed by Djellout, Guillin and 
Wu Section 3]. We have improved upon P by obtaining LSI (a), hence T 2 (a), under 
the spectral radius condition p < 1, where a is independent of the size n of the considered 
sample and the size of the matrices. 



5. Logarithmic Sobolev inequality for weakly dependent processes 

In this section we consider a stochastic process (£j)j =1 , with state space R m and initial 
distribution which is not necessarily Markovian; we also assume that the transition 
kernels have positive densities with respect to Lebesgue measure, and write 

dpj = pj(dxj | x^'V) = e~ u ^ xl) ^dxj (j = 2, . . . , n). 
The coupling between variables is measured by the following integral 

A jjfc (s) = sup f expf (s, ^(x^))) Pjidxj | a; '" 1 )), (s e M m , 1 < k < j < n) 

where as above d/dxk denotes the gradient with respect to Xk G W 11 . The main result in 
this section is the following. 

Theorem 5.1. Suppose that there exist constants a > and Kj^ > for 1 < k < j < n 
such that 

(i) and p k {. | x {k - l) ) (k = 2,...,n; x {k ~ 1] E M m(fc_1) ) satisfy LSI(a); 

(ii) A j>k (s) < exp(K jik ||s|| 2 /2) holds for all s G M m . 
Then the joint distribution P^ satisfies LSI(a n ) with 

n—2 n—1 j 

«n=Y^(i+^ n ( i + K ™)y (5.i) 

fc=0 m=k+l 

lie > 0, where Kj = (1 + e~ l ) YjeZo K n -t,n~j/a for j = 1, . . . , n - 1. 

Suppose further that there exist R > and pi > for £ = 1, . . . , n — 1 such that 
(Hi) Kj j, < pj_k for 1 < k < j < n, and Ym=i \/~Pt — 
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Then P^ satisfies LSI(a n ) where 

' (v^-V^) : 



ft 



n(n + 



l)(e - 1) 
it — ft 



R) e(n + l 



z/ R < a, 
if R = ft, 

i/ -R > ft. 



Before proving this theorem, we give simple sufficient conditions for hypothesis (ii) to 
hold. When m — 1, hypothesis (i) is equivalent to a condition on the cumulative distribu- 
tion functions by the criterion for LSI given in j^j. 

Proposition 5.2. In the above notation, let 1 < k < j and suppose that there exist a > 
and Lj^ > such that 

(i) pj(. | xV-V) satisfies GCQ/a) for all G R"^'- 1 ); 

(ii) Uj is twice continuously differentiable and the off-diagonal blocks of its Hessian ma- 
trix satisfy 

d 2 U; 



dxjdxk 



< L 



3,k 



as matrices 
Then 



Dm l)2\ 



(R' 



m (}2\ 



A jtk (s) < exp(L 2 ;fc ||s|| 2 /(2a)) (s G R m ). 



Proof of Proposition^^ Letting s = \\s\\ e for some unit vector e, we note that by (ii) the 
real function Xj i— > (e,duj/dxk) is Lj^-Lipschitz in the variable of integration, and that 



dxk 



Pj(d. 



(j-i) 



)> = 



since Pj( 



(i-i) 



) is a probability measure. Then, by (i), 



£exp(( S , |^)) p i (da; i | x^) < exp(«L 2 fe ||s|| 2 /2) 
holds for all sc^" -1 ) in R m(j_1 ). This inequality implies the Proposition. 



□ 



Proof of Proposition^^ For notational convenience, X denotes the state space R m . Then 
let / : X n — > R be a smooth and compactly supported function, and let gj : X n_J — > R be 
defined by g = f and by 

1/2 

, ,11 -i- II' ' .' I 111 11:1 



(n-J')^ 



) Vn—jid^n—j+l | 



X 



(5.2) 



for j = 1, . . . , n - 1; finally, let g n be the constant (/ / 2 dP (n) ) 1/2 . 
From the recursive formula ()5.2|) one can easily verify the identity 

/ / 2 log(/ 2 // fdP^AdPW = fl [ 9 2 3 \og{g 2 /g 2 +l )dP^ 
J x n v Jx» y J=0 



(5.3) 
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which is crucial to the proof; indeed, it allows us to obtain the result from logarithmic 
Sobolev inequalities on X. 

By hypothesis (i), the measure dp n -j = p n -j(dx n -j | x^ - - 7-1 )) satisfies LSI (a), whence 



X 



< - 



ot J x ^dx n _ , 



d 9j \ 2 , 

1 dp n - 



(j = 0,...,n-l), (5.4) 



where for j = n — 1 we take dpi = P^(dxi). The next step is to express these derivatives 
in terms of the gradient of /, using the identity 



9j 



9 9j 



dx n . 



/ f 7h- — dpn ' • • dpn - J+1 ~ 9 X) / 

JX n ~i OX n -j Z J X j- 



du r 



2 WUj n 



91 aZT d,Vn-i ■ ■ ■ dpn-j+i (5.5) 



dx 



n-j 



which follows from the definition (|5.2j) of g? and that of p n -j- The integrals on the right- 
hand side of ()5.5|) will be bounded by the following Lemma. 

Lemma 5.3. Let 0<£<j<n — 1, and assume that hypothesis (ii) holds. Then 



du n . 
8x r 



dpn-i 



< ge+ife Kn-t, n -j J g1^og{gll g 2 l+1 ) dp n -^j 1 . (5.6) 



Proof of Lemma \5.<A By definition of A n _£ in _j, we have 

/ expf (s, n ~ e ) - log An-t^-^s) J dp n -i < 1 (s G X), 
Jx ^ OX n -j / 

and hence by the dual formula for relative entropy, as in jH p. 693], 

./ {( Sj ^^) ~ logAn - i ' n -^) 9 i d Pn-i< J gl^g{g 2 Jg 2 i+1 )dp n _ t . 
Then hypothesis (ii) of the Theorem ensures that 

( S ' / ^-i gj dp n _ e ) < Kn-^n-j g^ +1 + / #f log fef/fl-f+i) 

ii 0%n-j ^ JX 

and the stated result follows by optimizing this over s G M m . 



□ 



Conclusion of the Proof of Theorem 5.1. When we integrate ()5.6|) with respect to 
dp n -e-i . . . dp n -j + i, we deduce by the Cauchy-Schwarz inequality that 



gi 



,dUr 



dx 



-dp n _ e ...dp 



'n-j+l 



n-j 



< gj (2 n n -e,n-j / g\ log(g1/gj +1 )dp n - e . . . dp n - j+1 
v Jw- e 



1/2 
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Then, by integrating the square of ()5.5|) with respect to dP^ n " and making a further 
application of the Cauchy-Schwarz inequality, we obtain 







/ 


dx n —j 





df 


/ 


dx n —j 



'dP {n - j) < (1 + e) 



where e > is arbitrary and kg is given by 



4 



| ^2 Kn-i,n-j hej I 



(5.7) 



Jx n ~ e 



From ()5.7|) . which holds true for j = 1, . . . , n — 1, we first prove the general result given 
in (|5.ip . By (|5.4|) and the Cauchy-Schwarz inequality again, we obtain from (|5.7|) the 
crucial inequality 



/ij- < dj + Kj ^ fe m (j = 1, . . . , n - 1) 



m=0 



where we have let 



2(l + e) 



df 



l+e 



-l J- 1 

K 



a 



E 

£=0 



J n—t,n—j 



y d pW (j = 0,...,n-l), 
(j = l,...,n-l). 



Since /iq — <^o an d all terms are positive, the partial sums H k = Ylj=o hj satisfy the system 
of inequalities 

H k <d k + (l + K k )H k ^ (/c = l,...,n-l), 
with H Q < d . By induction, one can deduce that 

n— 2 n— 1 

Hn-X < d n -l + J2 dk II + 
fc=0 £=fc+l 

which in turn implies the bound 

n— 2 re— 1 n— 1 

#n-l< (l+E II 
fc=o £=fc+l 

By (15. H|) this is equivalent to the inequality 

./x™ ^x™ ' a k=0 i=k+1 Jx n 

Since / is arbitrary, this ensures that P^ satisfies LSI(a n ) with a n as in (|5.1jl . 



j=0 



n— 2 n— 1 
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(iii) The extra hypothesis (iii) enables us to strengthen the preceding inequalities, so 
(|5.7|) leads to the convolution-type inequality 

1 + e^ 1 ,? ~ 1 2 

hj < dj + — ^— (J2 VpJ 1 * Vhi) 

for j = 1, . . . , n — 1, and ho < do for j = 0. By summing over j we obtain 

h k 1 k J 1 rj 

1+5 x — ^ / x — > , r, — \ 



i=0 j=0 ' i=l £=0 

which implies by Young's convolution inequality that 

A? Jc ^ ^ 1_ ^ 2 ^ 

^ (£>/«) £*<■ 

j=0 j=0 £=1 1=0 

Now let Rj = (X^=i v^) 2 anc ^ -^i = X^=o^' then by induction one can prove that 

fc— 1 k _-y 

H k <D k + Y< D i II —^~ R * 

j=0 £=j + l 

for = 1, . . . , n — 1, and hence 

(i+n^r'V-n^f^v, ( 5. 8) 

since .D^ < -D n -i and i?j < i? by hypothesis (iii). We finally select e to make the bound 
(15. 8|) precise, according to the relative values of R and a. 

When R = 0, we recover LSI(a) for as expected, since here P( n > is the tensor 
product of its marginal distributions, which satisfy LSI (a). 

When < R < a, we chooser = ( \J{a/R) - > so that (l+e -1 )i2/a = \J{R/a) < 
1 and hence 

oo 



which by (15. 3j) and the definition of P^„_i and D n _i implies the inequality 

/ / 2 log(/ 2 // f 2 dpM)dpw< - 2 [ \\vffdpw. 

JX n V JX n 7 (y/Ot — V R) JX n 

When R > a, we choose e = n in (|5.8|) . obtaining 

g „_,<^( (;;V;)"(W-l )/ | |V/fd P<»>; 
a V I 1 + l/n){R/a) - 1 / Jx™ 

as above this leads to the stated result by (|5.3j) . This concludes the proof. □ 
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6. Logarithmic Sobolev inequalities for Markov processes 



The results of the preceding section simplify considerably when we have an homogeneous 
Markov process (£j)?=i with state space W m , as we shall now show. Suppose that the 
transition measure is p(dy \ x) = e~ u ^ x,v 'dy where u is a twice continuously differentiable 
function such that 

A(s | x) = 



exp 



du ^ 

dx 



, s - -r—( x ^y)))p( d y \x) <oo 



[s, x g 



Then Theorem 15 . II has the following consequence. 



Corollary 6.1. Suppose that there exist constants k > and a > such that: 

(i) andp(. | x) (x G M m ) satisfy LSI(a); 

(ii) A(s | x) < exp(K||s|| 2 /2) holds for all s, x G M. m . 

Then the joint law P^ n > of the first n variables satisfies LSI(a n ), where 



(v^-V 7 ^)' 

a 



a, 



n(n + l)(e - 1) 
a\ n k — a 



k) e(n + 1) 



if k < a, 
if k = a, 

if k > a. 



(6.1) 



Proof. In the notation of section 5, we have Uj(x^') = u(xj^i, Xj), so we can take fcjj^m — 
for m = 1, . . . , j — 2, and = k for j = 2, . . . , n; hence we can take p\ = k and pj = 

for j = 2,3,.... Now we can apply Theorem 15.11 (iii) and obtain the stated result with 
R = k in the various cases. (In fact ()5.7|) simplifies considerably for a Markov process, and 
hence one can obtain an easier direct proof of Corollary 16.11 ) □ 

Proof of Theorem M.iA By the mean- value theorem and hypothesis (ii) of Theorem II .31 the 
function y i— > (e,du/dx) is L-Lipschitz (IR m ,£ 2 ) — > K for any unit vector e in M m , and 
hence A(s | x) < exp(||s|| 2 L 2 /(2a)) holds for all s G M m as in Proposition 15.21 Hence we 
can take k = L 2 /a in Corollary 16. II and deduce Theorem 11.31 with the various values of the 
constant. □ 



Remarks 6.2. (i) Theorem 15 . 1 1 and Corollary 16 . II extend with suitable changes in notation 
when the state space is a connected C 1 -smooth Riemannian manifold X. The proofs reduce 
to calculations in local co-ordinate charts. McCann has shown that a locally Lipschitz 
function on X is differentiable except on a set that has zero Riemannian volume; so a 
L-Lipschitz condition on / : X — > R is essentially equivalent to ||V/|| < L. 

(ii) Corollary 6.1 is a natural refinement of Theorems 11.11 and 11.21 Indeed LSI (a) 
implies T s (a). Then, in the notation of the mentioned results, suppose that u is a twice 
continuously differentiable function with bounded second-order partial derivatives. Then, 
by Proposition 12. 2\ hypotheses (i) and (ii) of Corollary 16.11 together imply that the map 
x i— ► p(. | x) is (/t/a) 1 / 2 Lipschitz as a function M. m — > (Prob2 (M m ), W2), hence M. m 
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(Pretty (R m ), W s ) as in Theorems 1 1 . 1 1 or \l . 21 Similarly Proposition l2.2l ensures that Theorem 
15. H is a refinement of Theorem 12. II with, for s — 2, 



1 j=l r 

M < M 2 /a = - sup V / 



d Uj ' 



dxi 



Pj(d 



< 



1 J_1 



fe=l 



Note also the similarity between the constants in Theorem 12.11 (iii) and Theorem 15.11 (iii) 
when s = 2 and one rescales R suitably. In Example 16.31 we show these constants to be 
optimal. 



Example 6.3. (Ornstein-Uhlenbeck Process) We now show that the constants K n of The- 
orem ITTl for Theorem 13 .If iii)) and a n of Corollary 16. II have optimal growth in n. For this 
purpose we consider the real Ornstein-Uhlenbeck process conditioned to start at i G R, 
namely the solution to the Ito stochastic differential equation 

dz[ x) = -pZ ( t x) dt + dBf ] , (t > 0) 

where (B^) is a real standard Brownian motion starting at 0, and p G M. In financial 
modelling, OU processes with p < are used to model stock prices in a rising market 
(see |1UI p 26] for instance). More precisely we consider the discrete-time Markov process 
(6j)"=i defined by = Z^) where r > 0, and test the Gaussian concentration inequality 
with the 1-Lipschitz function F n : (R n , I 1 ) -> R defined by F n {x^) = Y?j=i x j- 
The exponential integral satisfies 

n 

exp(sF n (x (n) )) P {n \dx {n) ) = E exp(sF n (e (n) )) = E exp^s^Z^). (6-2) 

3=1 

This sum can be expressed in terms of the increments of the OU process 

n n n—1 n—j—1 

E z t = E «' + E E - <')■ 

j = l 1=1 jr'=0 4=0 

with 6 1 = e~ pT . Moreover one can integrate the stochastic differential equation and prove 
that (Z^ +1 ^ t — 6Z^}) < k < n _i are independent random variables each with N(0, a 2 ) dis- 
tribution, where a 2 = (1 — 8 2 )/(2p) when p ^ 0, and a 2 = t when p = 0. Hence the 
exponential integral ()6.2|) equals 

exp(sEF n (£ (n) ) + s 2 K n /2) 



(6.3) 



n— 1 n— j — 1 

ri Eex pKs 

i=l j=0 i=0 

rt— 1 n—j — 1 



it it ± it j ^ 

where 



. . , . . . . 2 

K„ = a 2 



E(E*' 

7=0 i=0 
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However, hypothesis (i) of Theorem 11.11 holds with L = 9, since with distribution 
N(x, a 2 ) and p(. \ x) with distribution N{9x,a 2 ) satisfy GC(ki) where /ti = cr 2 , while 
hypothesis (ii) is satisfied with 

| x),p(. | x')) = W 1 (N(9x,a 2 ),N(9x',a 2 )) = 6\x - x'\ (x,x' e E). (6.4) 

Hence the constant n n {L) given by Theorem 1.1 is exactly the directly computed constant 
n n in (|6.3|) . in each of the cases L = 1, L > 1 and L < 1, corresponding to p = 0, p < 
and p > 0. 

As regards Corollary 16.11 note that the transition probability is given by 

pidy\x) = -^exp{-!^L)dy 

since is distributed as &c + B, . Hence by direct calculation we have 

1 6 2 T 9 

a = —, K = — , L = — ; 

<T Z <T^ <T Z 

consequently the dependence parameters (n/a) 1 ^ 2 and 6 1 given in ()6.4|) coincide, as in 
Remark I6.2f ii). 

Further, by considering the function f(x^) = exp(^" =1 O^Xj^j, one can prove that the 
joint law cannot satisfy a logarithmic Sobolev inequality with a n greater than some 
constant multiple of n~ 3 for 9 = 1, and (a/n) n for 9 > 1. Thus for 9 > 1, we recover 
the order of growth in n of the constants given in Corollary 16. 1\ whereas for 9 < 1, the 
constant given in Corollary 16.11 is independent of n. 

The OU process does not satisfy the Doeblin condition D , as Rosenblatt observes; see 
EH P- 214]. 
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